{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-01T00:09:09.469187Z",
     "start_time": "2020-07-01T00:09:05.792392Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.1.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-01T00:29:38.731608Z",
     "start_time": "2020-07-01T00:29:38.724627Z"
    }
   },
   "source": [
    "# 5.2 填充和步幅\n",
    "\n",
    "在上一节的例子里，我们使用高和宽为3的输入与高和宽为2的卷积核得到高和宽为2的输出。一般来说，假设输入形状是$n_h\\times n_w$，卷积核窗口形状是$k_h\\times k_w$，那么输出形状将会是\n",
    "\n",
    "$$(n_h-k_h+1) \\times (n_w-k_w+1).$$\n",
    "\n",
    "所以卷积层的输出形状由输入形状和卷积核窗口形状决定。本节我们将介绍卷积层的两个超参数，即填充和步幅。它们可以对给定形状的输入和卷积核改变输出形状。\n",
    "\n",
    "## 5.2.1 填充\n",
    "\n",
    "填充（padding）是指在输入高和宽的两侧填充元素（通常是0元素）。图5.2里我们在原输入高和宽的两侧分别添加了值为0的元素，使得输入高和宽从3变成了5，并导致输出高和宽由2增加到4。图5.2中的阴影部分为第一个输出元素及其计算所使用的输入和核数组元素：$0\\times0+0\\times1+0\\times2+0\\times3=0$。\n",
    "\n",
    "<div align=center>\n",
    "<img  src=\"../img/chapter05/5.2_conv_pad.svg\"/>\n",
    "</div>\n",
    "<div align=center>图5.2 在输入的高和宽两侧分别填充了0元素的二维互相关计算</div>\n",
    "\n",
    "\n",
    "一般来说，如果在高的两侧一共填充$p_h$行，在宽的两侧一共填充$p_w$列，那么输出形状将会是\n",
    "\n",
    "$$(n_h-k_h+p_h+1)\\times(n_w-k_w+p_w+1),$$\n",
    "\n",
    "也就是说，输出的高和宽会分别增加$p_h$和$p_w$。\n",
    "\n",
    "在很多情况下，我们会设置$p_h=k_h-1$和$p_w=k_w-1$来使输入和输出具有相同的高和宽。这样会方便在构造网络时推测每个层的输出形状。假设这里$k_h$是奇数，我们会在高的两侧分别填充$p_h/2$行。如果$k_h$是偶数，一种可能是在输入的顶端一侧填充$\\lceil p_h/2\\rceil$行，而在底端一侧填充$\\lfloor p_h/2\\rfloor$行。在宽的两侧填充同理。\n",
    "\n",
    "卷积神经网络经常使用奇数高宽的卷积核，如1、3、5和7，所以两端上的填充个数相等。对任意的二维数组`X`，设它的第`i`行第`j`列的元素为`X[i,j]`。当两端上的填充个数相等，并使输入和输出具有相同的高和宽时，我们就知道输出`Y[i,j]`是由输入以`X[i,j]`为中心的窗口同卷积核进行互相关计算得到的。\n",
    "\n",
    "下面的例子里我们创建一个高和宽为3的二维卷积层，然后设输入高和宽两侧的填充数分别为1。给定一个高和宽为8的输入，我们发现输出的高和宽也是8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-01T00:37:14.701935Z",
     "start_time": "2020-07-01T00:37:14.548991Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([8, 8])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 定义一个函数来计算卷积层。它初始化卷积层权重，并对输入和输出做相应的升维和降维\n",
    "def comp_conv2d(conv2d, X):\n",
    "    #input_shape = (samples, rows, cols, channels)\n",
    "    X = tf.reshape(X, (1,) + X.shape + (1,))\n",
    "    Y = conv2d(X)\n",
    "    # 排除不关系的第一维和最后一维\n",
    "    return tf.reshape(Y, Y.shape[1:3])\n",
    "\n",
    "conv2d = tf.keras.layers.Conv2D(1, kernel_size=3, padding='same')\n",
    "X = tf.random.uniform(shape=(8,8))\n",
    "# 注意这里是两侧分别填充1行或列，所以在两侧一共填充2行或列\n",
    "comp_conv2d(conv2d,X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.2 步幅\n",
    "\n",
    "在上一节里我们介绍了二维互相关运算。卷积窗口从输入数组的最左上方开始，按从左往右、从上往下的顺序，依次在输入数组上滑动。我们将每次滑动的行数和列数称为步幅（stride）。\n",
    "\n",
    "目前我们看到的例子里，在高和宽两个方向上步幅均为1。我们也可以使用更大步幅。图5.3展示了在高上步幅为3、在宽上步幅为2的二维互相关运算。可以看到，输出第一列第二个元素时，卷积窗口向下滑动了3行，而在输出第一行第二个元素时卷积窗口向右滑动了2列。当卷积窗口在输入上再向右滑动2列时，由于输入元素无法填满窗口，无结果输出。图5.3中的阴影部分为输出元素及其计算所使用的输入和核数组元素：$0\\times0+0\\times1+1\\times2+2\\times3=8$、$0\\times0+6\\times1+0\\times2+0\\times3=6$。\n",
    "\n",
    "![高和宽上步幅分别为3和2的二维互相关运算](../img/chapter05/5.2_conv_stride.svg)\n",
    "\n",
    "一般来说，当高上步幅为$s_h$，宽上步幅为$s_w$时，输出形状为\n",
    "\n",
    "$$\\lfloor(n_h-k_h+p_h+s_h)/s_h\\rfloor \\times \\lfloor(n_w-k_w+p_w+s_w)/s_w\\rfloor.$$\n",
    "\n",
    "如果设置$p_h=k_h-1$和$p_w=k_w-1$，那么输出形状将简化为$\\lfloor(n_h+s_h-1)/s_h\\rfloor \\times \\lfloor(n_w+s_w-1)/s_w\\rfloor$。更进一步，如果输入的高和宽能分别被高和宽上的步幅整除，那么输出形状将是$(n_h/s_h) \\times (n_w/s_w)$。\n",
    "\n",
    "下面我们令高和宽上的步幅均为2，从而使输入的高和宽减半。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# padding 参数解释 https://blog.csdn.net/wuzqChom/article/details/74785643\n",
    "conv2d = tf.keras.layers.Conv2D(1, kernel_size=3, padding='same', strides=2)\n",
    "comp_conv2d(conv2d,X).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-01T00:47:05.349134Z",
     "start_time": "2020-07-01T00:47:05.331207Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class Conv2D in module tensorflow.python.keras.layers.convolutional:\n",
      "\n",
      "class Conv2D(Conv)\n",
      " |  2D convolution layer (e.g. spatial convolution over images).\n",
      " |  \n",
      " |  This layer creates a convolution kernel that is convolved\n",
      " |  with the layer input to produce a tensor of\n",
      " |  outputs. If `use_bias` is True,\n",
      " |  a bias vector is created and added to the outputs. Finally, if\n",
      " |  `activation` is not `None`, it is applied to the outputs as well.\n",
      " |  \n",
      " |  When using this layer as the first layer in a model,\n",
      " |  provide the keyword argument `input_shape`\n",
      " |  (tuple of integers, does not include the sample axis),\n",
      " |  e.g. `input_shape=(128, 128, 3)` for 128x128 RGB pictures\n",
      " |  in `data_format=\"channels_last\"`.\n",
      " |  \n",
      " |  Arguments:\n",
      " |    filters: Integer, the dimensionality of the output space\n",
      " |      (i.e. the number of output filters in the convolution).\n",
      " |    kernel_size: An integer or tuple/list of 2 integers, specifying the\n",
      " |      height and width of the 2D convolution window.\n",
      " |      Can be a single integer to specify the same value for\n",
      " |      all spatial dimensions.\n",
      " |    strides: An integer or tuple/list of 2 integers,\n",
      " |      specifying the strides of the convolution along the height and width.\n",
      " |      Can be a single integer to specify the same value for\n",
      " |      all spatial dimensions.\n",
      " |      Specifying any stride value != 1 is incompatible with specifying\n",
      " |      any `dilation_rate` value != 1.\n",
      " |    padding: one of `\"valid\"` or `\"same\"` (case-insensitive).\n",
      " |    data_format: A string,\n",
      " |      one of `channels_last` (default) or `channels_first`.\n",
      " |      The ordering of the dimensions in the inputs.\n",
      " |      `channels_last` corresponds to inputs with shape\n",
      " |      `(batch, height, width, channels)` while `channels_first`\n",
      " |      corresponds to inputs with shape\n",
      " |      `(batch, channels, height, width)`.\n",
      " |      It defaults to the `image_data_format` value found in your\n",
      " |      Keras config file at `~/.keras/keras.json`.\n",
      " |      If you never set it, then it will be \"channels_last\".\n",
      " |    dilation_rate: an integer or tuple/list of 2 integers, specifying\n",
      " |      the dilation rate to use for dilated convolution.\n",
      " |      Can be a single integer to specify the same value for\n",
      " |      all spatial dimensions.\n",
      " |      Currently, specifying any `dilation_rate` value != 1 is\n",
      " |      incompatible with specifying any stride value != 1.\n",
      " |    activation: Activation function to use.\n",
      " |      If you don't specify anything, no activation is applied\n",
      " |      (ie. \"linear\" activation: `a(x) = x`).\n",
      " |    use_bias: Boolean, whether the layer uses a bias vector.\n",
      " |    kernel_initializer: Initializer for the `kernel` weights matrix.\n",
      " |    bias_initializer: Initializer for the bias vector.\n",
      " |    kernel_regularizer: Regularizer function applied to\n",
      " |      the `kernel` weights matrix.\n",
      " |    bias_regularizer: Regularizer function applied to the bias vector.\n",
      " |    activity_regularizer: Regularizer function applied to\n",
      " |      the output of the layer (its \"activation\")..\n",
      " |    kernel_constraint: Constraint function applied to the kernel matrix.\n",
      " |    bias_constraint: Constraint function applied to the bias vector.\n",
      " |  \n",
      " |  Input shape:\n",
      " |    4D tensor with shape:\n",
      " |    `(samples, channels, rows, cols)` if data_format='channels_first'\n",
      " |    or 4D tensor with shape:\n",
      " |    `(samples, rows, cols, channels)` if data_format='channels_last'.\n",
      " |  \n",
      " |  Output shape:\n",
      " |    4D tensor with shape:\n",
      " |    `(samples, filters, new_rows, new_cols)` if data_format='channels_first'\n",
      " |    or 4D tensor with shape:\n",
      " |    `(samples, new_rows, new_cols, filters)` if data_format='channels_last'.\n",
      " |    `rows` and `cols` values might have changed due to padding.\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      Conv2D\n",
      " |      Conv\n",
      " |      tensorflow.python.keras.engine.base_layer.Layer\n",
      " |      tensorflow.python.module.module.Module\n",
      " |      tensorflow.python.training.tracking.tracking.AutoTrackable\n",
      " |      tensorflow.python.training.tracking.base.Trackable\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, **kwargs)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from Conv:\n",
      " |  \n",
      " |  build(self, input_shape)\n",
      " |      Creates the variables of the layer (optional, for subclass implementers).\n",
      " |      \n",
      " |      This is a method that implementers of subclasses of `Layer` or `Model`\n",
      " |      can override if they need a state-creation step in-between\n",
      " |      layer instantiation and layer call.\n",
      " |      \n",
      " |      This is typically used to create the weights of `Layer` subclasses.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        input_shape: Instance of `TensorShape`, or list of instances of\n",
      " |          `TensorShape` if the layer expects a list of inputs\n",
      " |          (one instance per input).\n",
      " |  \n",
      " |  call(self, inputs)\n",
      " |      This is where the layer's logic lives.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          inputs: Input tensor, or list/tuple of input tensors.\n",
      " |          **kwargs: Additional keyword arguments.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A tensor or list/tuple of tensors.\n",
      " |  \n",
      " |  compute_output_shape(self, input_shape)\n",
      " |      Computes the output shape of the layer.\n",
      " |      \n",
      " |      If the layer has not been built, this method will call `build` on the\n",
      " |      layer. This assumes that the layer will later be used with inputs that\n",
      " |      match the input shape provided here.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          input_shape: Shape tuple (tuple of integers)\n",
      " |              or list of shape tuples (one per output tensor of the layer).\n",
      " |              Shape tuples can include None for free dimensions,\n",
      " |              instead of an integer.\n",
      " |      \n",
      " |      Returns:\n",
      " |          An input shape tuple.\n",
      " |  \n",
      " |  get_config(self)\n",
      " |      Returns the config of the layer.\n",
      " |      \n",
      " |      A layer config is a Python dictionary (serializable)\n",
      " |      containing the configuration of a layer.\n",
      " |      The same layer can be reinstantiated later\n",
      " |      (without its trained weights) from this configuration.\n",
      " |      \n",
      " |      The config of a layer does not include connectivity\n",
      " |      information, nor the layer class name. These are handled\n",
      " |      by `Network` (one layer of abstraction above).\n",
      " |      \n",
      " |      Returns:\n",
      " |          Python dictionary.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from tensorflow.python.keras.engine.base_layer.Layer:\n",
      " |  \n",
      " |  __call__(self, inputs, *args, **kwargs)\n",
      " |      Wraps `call`, applying pre- and post-processing steps.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        inputs: input tensor(s).\n",
      " |        *args: additional positional arguments to be passed to `self.call`.\n",
      " |        **kwargs: additional keyword arguments to be passed to `self.call`.\n",
      " |      \n",
      " |      Returns:\n",
      " |        Output tensor(s).\n",
      " |      \n",
      " |      Note:\n",
      " |        - The following optional keyword arguments are reserved for specific uses:\n",
      " |          * `training`: Boolean scalar tensor of Python boolean indicating\n",
      " |            whether the `call` is meant for training or inference.\n",
      " |          * `mask`: Boolean input mask.\n",
      " |        - If the layer's `call` method takes a `mask` argument (as some Keras\n",
      " |          layers do), its default value will be set to the mask generated\n",
      " |          for `inputs` by the previous layer (if `input` did come from\n",
      " |          a layer that generated a corresponding mask, i.e. if it came from\n",
      " |          a Keras layer with masking support.\n",
      " |      \n",
      " |      Raises:\n",
      " |        ValueError: if the layer's `call` method returns None (an invalid value).\n",
      " |  \n",
      " |  __delattr__(self, name)\n",
      " |      Implement delattr(self, name).\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __setattr__(self, name, value)\n",
      " |      Support self.foo = trackable syntax.\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  add_loss(self, losses, inputs=None)\n",
      " |      Add loss tensor(s), potentially dependent on layer inputs.\n",
      " |      \n",
      " |      Some losses (for instance, activity regularization losses) may be dependent\n",
      " |      on the inputs passed when calling a layer. Hence, when reusing the same\n",
      " |      layer on different inputs `a` and `b`, some entries in `layer.losses` may\n",
      " |      be dependent on `a` and some on `b`. This method automatically keeps track\n",
      " |      of dependencies.\n",
      " |      \n",
      " |      This method can be used inside a subclassed layer or model's `call`\n",
      " |      function, in which case `losses` should be a Tensor or list of Tensors.\n",
      " |      \n",
      " |      Example:\n",
      " |      \n",
      " |      ```python\n",
      " |      class MyLayer(tf.keras.layers.Layer):\n",
      " |        def call(inputs, self):\n",
      " |          self.add_loss(tf.abs(tf.reduce_mean(inputs)), inputs=True)\n",
      " |          return inputs\n",
      " |      ```\n",
      " |      \n",
      " |      This method can also be called directly on a Functional Model during\n",
      " |      construction. In this case, any loss Tensors passed to this Model must\n",
      " |      be symbolic and be able to be traced back to the model's `Input`s. These\n",
      " |      losses become part of the model's topology and are tracked in `get_config`.\n",
      " |      \n",
      " |      Example:\n",
      " |      \n",
      " |      ```python\n",
      " |      inputs = tf.keras.Input(shape=(10,))\n",
      " |      x = tf.keras.layers.Dense(10)(inputs)\n",
      " |      outputs = tf.keras.layers.Dense(1)(x)\n",
      " |      model = tf.keras.Model(inputs, outputs)\n",
      " |      # Actvity regularization.\n",
      " |      model.add_loss(tf.abs(tf.reduce_mean(x)))\n",
      " |      ```\n",
      " |      \n",
      " |      If this is not the case for your loss (if, for example, your loss references\n",
      " |      a `Variable` of one of the model's layers), you can wrap your loss in a\n",
      " |      zero-argument lambda. These losses are not tracked as part of the model's\n",
      " |      topology since they can't be serialized.\n",
      " |      \n",
      " |      Example:\n",
      " |      \n",
      " |      ```python\n",
      " |      inputs = tf.keras.Input(shape=(10,))\n",
      " |      x = tf.keras.layers.Dense(10)(inputs)\n",
      " |      outputs = tf.keras.layers.Dense(1)(x)\n",
      " |      model = tf.keras.Model(inputs, outputs)\n",
      " |      # Weight regularization.\n",
      " |      model.add_loss(lambda: tf.reduce_mean(x.kernel))\n",
      " |      ```\n",
      " |      \n",
      " |      The `get_losses_for` method allows to retrieve the losses relevant to a\n",
      " |      specific set of inputs.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        losses: Loss tensor, or list/tuple of tensors. Rather than tensors, losses\n",
      " |          may also be zero-argument callables which create a loss tensor.\n",
      " |        inputs: Ignored when executing eagerly. If anything other than None is\n",
      " |          passed, it signals the losses are conditional on some of the layer's\n",
      " |          inputs, and thus they should only be run where these inputs are\n",
      " |          available. This is the case for activity regularization losses, for\n",
      " |          instance. If `None` is passed, the losses are assumed\n",
      " |          to be unconditional, and will apply across all dataflows of the layer\n",
      " |          (e.g. weight regularization losses).\n",
      " |  \n",
      " |  add_metric(self, value, aggregation=None, name=None)\n",
      " |      Adds metric tensor to the layer.\n",
      " |      \n",
      " |      Args:\n",
      " |        value: Metric tensor.\n",
      " |        aggregation: Sample-wise metric reduction function. If `aggregation=None`,\n",
      " |          it indicates that the metric tensor provided has been aggregated\n",
      " |          already. eg, `bin_acc = BinaryAccuracy(name='acc')` followed by\n",
      " |          `model.add_metric(bin_acc(y_true, y_pred))`. If aggregation='mean', the\n",
      " |          given metric tensor will be sample-wise reduced using `mean` function.\n",
      " |          eg, `model.add_metric(tf.reduce_sum(outputs), name='output_mean',\n",
      " |          aggregation='mean')`.\n",
      " |        name: String metric name.\n",
      " |      \n",
      " |      Raises:\n",
      " |        ValueError: If `aggregation` is anything other than None or `mean`.\n",
      " |  \n",
      " |  add_update(self, updates, inputs=None)\n",
      " |      Add update op(s), potentially dependent on layer inputs. (deprecated arguments)\n",
      " |      \n",
      " |      Warning: SOME ARGUMENTS ARE DEPRECATED: `(inputs)`. They will be removed in a future version.\n",
      " |      Instructions for updating:\n",
      " |      `inputs` is now automatically inferred\n",
      " |      \n",
      " |      Weight updates (for instance, the updates of the moving mean and variance\n",
      " |      in a BatchNormalization layer) may be dependent on the inputs passed\n",
      " |      when calling a layer. Hence, when reusing the same layer on\n",
      " |      different inputs `a` and `b`, some entries in `layer.updates` may be\n",
      " |      dependent on `a` and some on `b`. This method automatically keeps track\n",
      " |      of dependencies.\n",
      " |      \n",
      " |      The `get_updates_for` method allows to retrieve the updates relevant to a\n",
      " |      specific set of inputs.\n",
      " |      \n",
      " |      This call is ignored when eager execution is enabled (in that case, variable\n",
      " |      updates are run on the fly and thus do not need to be tracked for later\n",
      " |      execution).\n",
      " |      \n",
      " |      Arguments:\n",
      " |        updates: Update op, or list/tuple of update ops, or zero-arg callable\n",
      " |          that returns an update op. A zero-arg callable should be passed in\n",
      " |          order to disable running the updates by setting `trainable=False`\n",
      " |          on this Layer, when executing in Eager mode.\n",
      " |        inputs: Deprecated, will be automatically inferred.\n",
      " |  \n",
      " |  add_variable(self, *args, **kwargs)\n",
      " |      Deprecated, do NOT use! Alias for `add_weight`. (deprecated)\n",
      " |      \n",
      " |      Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version.\n",
      " |      Instructions for updating:\n",
      " |      Please use `layer.add_weight` method instead.\n",
      " |  \n",
      " |  add_weight(self, name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, partitioner=None, use_resource=None, synchronization=<VariableSynchronization.AUTO: 0>, aggregation=<VariableAggregation.NONE: 0>, **kwargs)\n",
      " |      Adds a new variable to the layer.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        name: Variable name.\n",
      " |        shape: Variable shape. Defaults to scalar if unspecified.\n",
      " |        dtype: The type of the variable. Defaults to `self.dtype` or `float32`.\n",
      " |        initializer: Initializer instance (callable).\n",
      " |        regularizer: Regularizer instance (callable).\n",
      " |        trainable: Boolean, whether the variable should be part of the layer's\n",
      " |          \"trainable_variables\" (e.g. variables, biases)\n",
      " |          or \"non_trainable_variables\" (e.g. BatchNorm mean and variance).\n",
      " |          Note that `trainable` cannot be `True` if `synchronization`\n",
      " |          is set to `ON_READ`.\n",
      " |        constraint: Constraint instance (callable).\n",
      " |        partitioner: Partitioner to be passed to the `Trackable` API.\n",
      " |        use_resource: Whether to use `ResourceVariable`.\n",
      " |        synchronization: Indicates when a distributed a variable will be\n",
      " |          aggregated. Accepted values are constants defined in the class\n",
      " |          `tf.VariableSynchronization`. By default the synchronization is set to\n",
      " |          `AUTO` and the current `DistributionStrategy` chooses\n",
      " |          when to synchronize. If `synchronization` is set to `ON_READ`,\n",
      " |          `trainable` must not be set to `True`.\n",
      " |        aggregation: Indicates how a distributed variable will be aggregated.\n",
      " |          Accepted values are constants defined in the class\n",
      " |          `tf.VariableAggregation`.\n",
      " |        **kwargs: Additional keyword arguments. Accepted values are `getter`,\n",
      " |          `collections`, `experimental_autocast` and `caching_device`.\n",
      " |      \n",
      " |      Returns:\n",
      " |        The created variable. Usually either a `Variable` or `ResourceVariable`\n",
      " |        instance. If `partitioner` is not `None`, a `PartitionedVariable`\n",
      " |        instance is returned.\n",
      " |      \n",
      " |      Raises:\n",
      " |        RuntimeError: If called with partitioned variable regularization and\n",
      " |          eager execution is enabled.\n",
      " |        ValueError: When giving unsupported dtype and no initializer or when\n",
      " |          trainable has been set to True with synchronization set as `ON_READ`.\n",
      " |  \n",
      " |  apply(self, inputs, *args, **kwargs)\n",
      " |      Deprecated, do NOT use! (deprecated)\n",
      " |      \n",
      " |      Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version.\n",
      " |      Instructions for updating:\n",
      " |      Please use `layer.__call__` method instead.\n",
      " |      \n",
      " |      This is an alias of `self.__call__`.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        inputs: Input tensor(s).\n",
      " |        *args: additional positional arguments to be passed to `self.call`.\n",
      " |        **kwargs: additional keyword arguments to be passed to `self.call`.\n",
      " |      \n",
      " |      Returns:\n",
      " |        Output tensor(s).\n",
      " |  \n",
      " |  compute_mask(self, inputs, mask=None)\n",
      " |      Computes an output mask tensor.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          inputs: Tensor or list of tensors.\n",
      " |          mask: Tensor or list of tensors.\n",
      " |      \n",
      " |      Returns:\n",
      " |          None or a tensor (or list of tensors,\n",
      " |              one per output tensor of the layer).\n",
      " |  \n",
      " |  compute_output_signature(self, input_signature)\n",
      " |      Compute the output tensor signature of the layer based on the inputs.\n",
      " |      \n",
      " |      Unlike a TensorShape object, a TensorSpec object contains both shape\n",
      " |      and dtype information for a tensor. This method allows layers to provide\n",
      " |      output dtype information if it is different from the input dtype.\n",
      " |      For any layer that doesn't implement this function,\n",
      " |      the framework will fall back to use `compute_output_shape`, and will\n",
      " |      assume that the output dtype matches the input dtype.\n",
      " |      \n",
      " |      Args:\n",
      " |        input_signature: Single TensorSpec or nested structure of TensorSpec\n",
      " |          objects, describing a candidate input for the layer.\n",
      " |      \n",
      " |      Returns:\n",
      " |        Single TensorSpec or nested structure of TensorSpec objects, describing\n",
      " |          how the layer would transform the provided input.\n",
      " |      \n",
      " |      Raises:\n",
      " |        TypeError: If input_signature contains a non-TensorSpec object.\n",
      " |  \n",
      " |  count_params(self)\n",
      " |      Count the total number of scalars composing the weights.\n",
      " |      \n",
      " |      Returns:\n",
      " |          An integer count.\n",
      " |      \n",
      " |      Raises:\n",
      " |          ValueError: if the layer isn't yet built\n",
      " |            (in which case its weights aren't yet defined).\n",
      " |  \n",
      " |  get_input_at(self, node_index)\n",
      " |      Retrieves the input tensor(s) of a layer at a given node.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          node_index: Integer, index of the node\n",
      " |              from which to retrieve the attribute.\n",
      " |              E.g. `node_index=0` will correspond to the\n",
      " |              first time the layer was called.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A tensor (or list of tensors if the layer has multiple inputs).\n",
      " |      \n",
      " |      Raises:\n",
      " |        RuntimeError: If called in Eager mode.\n",
      " |  \n",
      " |  get_input_mask_at(self, node_index)\n",
      " |      Retrieves the input mask tensor(s) of a layer at a given node.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          node_index: Integer, index of the node\n",
      " |              from which to retrieve the attribute.\n",
      " |              E.g. `node_index=0` will correspond to the\n",
      " |              first time the layer was called.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A mask tensor\n",
      " |          (or list of tensors if the layer has multiple inputs).\n",
      " |  \n",
      " |  get_input_shape_at(self, node_index)\n",
      " |      Retrieves the input shape(s) of a layer at a given node.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          node_index: Integer, index of the node\n",
      " |              from which to retrieve the attribute.\n",
      " |              E.g. `node_index=0` will correspond to the\n",
      " |              first time the layer was called.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A shape tuple\n",
      " |          (or list of shape tuples if the layer has multiple inputs).\n",
      " |      \n",
      " |      Raises:\n",
      " |        RuntimeError: If called in Eager mode.\n",
      " |  \n",
      " |  get_losses_for(self, inputs)\n",
      " |      Retrieves losses relevant to a specific set of inputs.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        inputs: Input tensor or list/tuple of input tensors.\n",
      " |      \n",
      " |      Returns:\n",
      " |        List of loss tensors of the layer that depend on `inputs`.\n",
      " |  \n",
      " |  get_output_at(self, node_index)\n",
      " |      Retrieves the output tensor(s) of a layer at a given node.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          node_index: Integer, index of the node\n",
      " |              from which to retrieve the attribute.\n",
      " |              E.g. `node_index=0` will correspond to the\n",
      " |              first time the layer was called.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A tensor (or list of tensors if the layer has multiple outputs).\n",
      " |      \n",
      " |      Raises:\n",
      " |        RuntimeError: If called in Eager mode.\n",
      " |  \n",
      " |  get_output_mask_at(self, node_index)\n",
      " |      Retrieves the output mask tensor(s) of a layer at a given node.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          node_index: Integer, index of the node\n",
      " |              from which to retrieve the attribute.\n",
      " |              E.g. `node_index=0` will correspond to the\n",
      " |              first time the layer was called.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A mask tensor\n",
      " |          (or list of tensors if the layer has multiple outputs).\n",
      " |  \n",
      " |  get_output_shape_at(self, node_index)\n",
      " |      Retrieves the output shape(s) of a layer at a given node.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          node_index: Integer, index of the node\n",
      " |              from which to retrieve the attribute.\n",
      " |              E.g. `node_index=0` will correspond to the\n",
      " |              first time the layer was called.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A shape tuple\n",
      " |          (or list of shape tuples if the layer has multiple outputs).\n",
      " |      \n",
      " |      Raises:\n",
      " |        RuntimeError: If called in Eager mode.\n",
      " |  \n",
      " |  get_updates_for(self, inputs)\n",
      " |      Retrieves updates relevant to a specific set of inputs.\n",
      " |      \n",
      " |      Arguments:\n",
      " |        inputs: Input tensor or list/tuple of input tensors.\n",
      " |      \n",
      " |      Returns:\n",
      " |        List of update ops of the layer that depend on `inputs`.\n",
      " |  \n",
      " |  get_weights(self)\n",
      " |      Returns the current weights of the layer.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Weights values as a list of numpy arrays.\n",
      " |  \n",
      " |  set_weights(self, weights)\n",
      " |      Sets the weights of the layer, from Numpy arrays.\n",
      " |      \n",
      " |      Arguments:\n",
      " |          weights: a list of Numpy arrays. The number\n",
      " |              of arrays and their shape must match\n",
      " |              number of the dimensions of the weights\n",
      " |              of the layer (i.e. it should match the\n",
      " |              output of `get_weights`).\n",
      " |      \n",
      " |      Raises:\n",
      " |          ValueError: If the provided weights list does not match the\n",
      " |              layer's specifications.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Class methods inherited from tensorflow.python.keras.engine.base_layer.Layer:\n",
      " |  \n",
      " |  from_config(config) from builtins.type\n",
      " |      Creates a layer from its config.\n",
      " |      \n",
      " |      This method is the reverse of `get_config`,\n",
      " |      capable of instantiating the same layer from the config\n",
      " |      dictionary. It does not handle layer connectivity\n",
      " |      (handled by Network), nor weights (handled by `set_weights`).\n",
      " |      \n",
      " |      Arguments:\n",
      " |          config: A Python dictionary, typically the\n",
      " |              output of get_config.\n",
      " |      \n",
      " |      Returns:\n",
      " |          A layer instance.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from tensorflow.python.keras.engine.base_layer.Layer:\n",
      " |  \n",
      " |  activity_regularizer\n",
      " |      Optional regularizer function for the output of this layer.\n",
      " |  \n",
      " |  dtype\n",
      " |  \n",
      " |  dynamic\n",
      " |  \n",
      " |  inbound_nodes\n",
      " |      Deprecated, do NOT use! Only for compatibility with external Keras.\n",
      " |  \n",
      " |  input\n",
      " |      Retrieves the input tensor(s) of a layer.\n",
      " |      \n",
      " |      Only applicable if the layer has exactly one input,\n",
      " |      i.e. if it is connected to one incoming layer.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Input tensor or list of input tensors.\n",
      " |      \n",
      " |      Raises:\n",
      " |        RuntimeError: If called in Eager mode.\n",
      " |        AttributeError: If no inbound nodes are found.\n",
      " |  \n",
      " |  input_mask\n",
      " |      Retrieves the input mask tensor(s) of a layer.\n",
      " |      \n",
      " |      Only applicable if the layer has exactly one inbound node,\n",
      " |      i.e. if it is connected to one incoming layer.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Input mask tensor (potentially None) or list of input\n",
      " |          mask tensors.\n",
      " |      \n",
      " |      Raises:\n",
      " |          AttributeError: if the layer is connected to\n",
      " |          more than one incoming layers.\n",
      " |  \n",
      " |  input_shape\n",
      " |      Retrieves the input shape(s) of a layer.\n",
      " |      \n",
      " |      Only applicable if the layer has exactly one input,\n",
      " |      i.e. if it is connected to one incoming layer, or if all inputs\n",
      " |      have the same shape.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Input shape, as an integer shape tuple\n",
      " |          (or list of shape tuples, one tuple per input tensor).\n",
      " |      \n",
      " |      Raises:\n",
      " |          AttributeError: if the layer has no defined input_shape.\n",
      " |          RuntimeError: if called in Eager mode.\n",
      " |  \n",
      " |  input_spec\n",
      " |  \n",
      " |  losses\n",
      " |      Losses which are associated with this `Layer`.\n",
      " |      \n",
      " |      Variable regularization tensors are created when this property is accessed,\n",
      " |      so it is eager safe: accessing `losses` under a `tf.GradientTape` will\n",
      " |      propagate gradients back to the corresponding variables.\n",
      " |      \n",
      " |      Returns:\n",
      " |        A list of tensors.\n",
      " |  \n",
      " |  metrics\n",
      " |  \n",
      " |  name\n",
      " |      Returns the name of this module as passed or determined in the ctor.\n",
      " |      \n",
      " |      NOTE: This is not the same as the `self.name_scope.name` which includes\n",
      " |      parent module names.\n",
      " |  \n",
      " |  non_trainable_variables\n",
      " |  \n",
      " |  non_trainable_weights\n",
      " |  \n",
      " |  outbound_nodes\n",
      " |      Deprecated, do NOT use! Only for compatibility with external Keras.\n",
      " |  \n",
      " |  output\n",
      " |      Retrieves the output tensor(s) of a layer.\n",
      " |      \n",
      " |      Only applicable if the layer has exactly one output,\n",
      " |      i.e. if it is connected to one incoming layer.\n",
      " |      \n",
      " |      Returns:\n",
      " |        Output tensor or list of output tensors.\n",
      " |      \n",
      " |      Raises:\n",
      " |        AttributeError: if the layer is connected to more than one incoming\n",
      " |          layers.\n",
      " |        RuntimeError: if called in Eager mode.\n",
      " |  \n",
      " |  output_mask\n",
      " |      Retrieves the output mask tensor(s) of a layer.\n",
      " |      \n",
      " |      Only applicable if the layer has exactly one inbound node,\n",
      " |      i.e. if it is connected to one incoming layer.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Output mask tensor (potentially None) or list of output\n",
      " |          mask tensors.\n",
      " |      \n",
      " |      Raises:\n",
      " |          AttributeError: if the layer is connected to\n",
      " |          more than one incoming layers.\n",
      " |  \n",
      " |  output_shape\n",
      " |      Retrieves the output shape(s) of a layer.\n",
      " |      \n",
      " |      Only applicable if the layer has one output,\n",
      " |      or if all outputs have the same shape.\n",
      " |      \n",
      " |      Returns:\n",
      " |          Output shape, as an integer shape tuple\n",
      " |          (or list of shape tuples, one tuple per output tensor).\n",
      " |      \n",
      " |      Raises:\n",
      " |          AttributeError: if the layer has no defined output shape.\n",
      " |          RuntimeError: if called in Eager mode.\n",
      " |  \n",
      " |  stateful\n",
      " |  \n",
      " |  trainable\n",
      " |  \n",
      " |  trainable_variables\n",
      " |      Sequence of trainable variables owned by this module and its submodules.\n",
      " |      \n",
      " |      Note: this method uses reflection to find variables on the current instance\n",
      " |      and submodules. For performance reasons you may wish to cache the result\n",
      " |      of calling this method if you don't expect the return value to change.\n",
      " |      \n",
      " |      Returns:\n",
      " |        A sequence of variables for the current module (sorted by attribute\n",
      " |        name) followed by variables from all submodules recursively (breadth\n",
      " |        first).\n",
      " |  \n",
      " |  trainable_weights\n",
      " |  \n",
      " |  updates\n",
      " |  \n",
      " |  variables\n",
      " |      Returns the list of all layer variables/weights.\n",
      " |      \n",
      " |      Alias of `self.weights`.\n",
      " |      \n",
      " |      Returns:\n",
      " |        A list of variables.\n",
      " |  \n",
      " |  weights\n",
      " |      Returns the list of all layer variables/weights.\n",
      " |      \n",
      " |      Returns:\n",
      " |        A list of variables.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Class methods inherited from tensorflow.python.module.module.Module:\n",
      " |  \n",
      " |  with_name_scope(method) from builtins.type\n",
      " |      Decorator to automatically enter the module name scope.\n",
      " |      \n",
      " |      ```\n",
      " |      class MyModule(tf.Module):\n",
      " |        @tf.Module.with_name_scope\n",
      " |        def __call__(self, x):\n",
      " |          if not hasattr(self, 'w'):\n",
      " |            self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n",
      " |          return tf.matmul(x, self.w)\n",
      " |      ```\n",
      " |      \n",
      " |      Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose\n",
      " |      names included the module name:\n",
      " |      \n",
      " |      ```\n",
      " |      mod = MyModule()\n",
      " |      mod(tf.ones([8, 32]))\n",
      " |      # ==> <tf.Tensor: ...>\n",
      " |      mod.w\n",
      " |      # ==> <tf.Variable ...'my_module/w:0'>\n",
      " |      ```\n",
      " |      \n",
      " |      Args:\n",
      " |        method: The method to wrap.\n",
      " |      \n",
      " |      Returns:\n",
      " |        The original method wrapped such that it enters the module's name scope.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from tensorflow.python.module.module.Module:\n",
      " |  \n",
      " |  name_scope\n",
      " |      Returns a `tf.name_scope` instance for this class.\n",
      " |  \n",
      " |  submodules\n",
      " |      Sequence of all sub-modules.\n",
      " |      \n",
      " |      Submodules are modules which are properties of this module, or found as\n",
      " |      properties of modules which are properties of this module (and so on).\n",
      " |      \n",
      " |      ```\n",
      " |      a = tf.Module()\n",
      " |      b = tf.Module()\n",
      " |      c = tf.Module()\n",
      " |      a.b = b\n",
      " |      b.c = c\n",
      " |      assert list(a.submodules) == [b, c]\n",
      " |      assert list(b.submodules) == [c]\n",
      " |      assert list(c.submodules) == []\n",
      " |      ```\n",
      " |      \n",
      " |      Returns:\n",
      " |        A sequence of all submodules.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from tensorflow.python.training.tracking.base.Trackable:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(tf.keras.layers.Conv2D)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来是一个稍微复杂点儿的例子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-01T00:56:44.808738Z",
     "start_time": "2020-07-01T00:56:44.799760Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([2, 1])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conv2d = tf.keras.layers.Conv2D(1, kernel_size=(3,5), padding='valid', strides=(3,4))\n",
    "comp_conv2d(conv2d, X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 注：本节除了代码之外与原书基本相同，[原书传送门](https://zh.d2l.ai/chapter_convolutional-neural-networks/padding-and-strides.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
